Universal-to-Specific Framework for Complex Action Recognition

Video-based action recognition has recently attracted much attention in the field of computer vision. To solve more complex tasks, it become necessary to distinguish different levels interclass variations. Inspired by a common flowchart based on human decision-making process that first narrows down probable classes and then applies "rethinking" for finer-level recognition, we propose an effective universal-to-specific (U2S) framework recognition. The U2S is composed three subnetworks: universal network, category-specific mask network. network learns feature representations. generates masks confusing through category regularization output further used guide class-specific entire optimized end-to-end manner. Experiments variety benchmark datasets, e.g., Something-Something, UCF101, HMDB51 demonstrate effectiveness framework; i.e., can focus discriminative spatiotemporal regions categories. We visualize relationship between classes, showing indeed improves discriminability learned features. Moreover, proposed model general may adopt any base